A Multi-instance Multi-label Dual Learning Approach for Video Captioning

Video captioning is a challenging task in the field of multimedia processing, which aims to generate informative natural language descriptions/captions describe video contents. Previous approaches mainly focused on capturing visual information videos using an encoder-decoder structure captions. Recently, new encoder-decoder-reconstructor was proposed for captioning, captured both and Based this, this article proposes novel multi-instance multi-label dual learning approach (MIMLDL) captions based structure. Specifically, MIMLDL contains two modules: caption generation reconstruction modules. The module utilizes lexical fully convolutional neural network (Lexical FCN) with weakly supervised mechanism learn translatable mapping between regions labels Then synthesizes sequences reproduce raw outputs module. A fine-tunes modules according gap reproduced videos. Thus, our can minimize semantic generated by minimizing differences sequences. Experimental results benchmark dataset demonstrate that improve accuracy captioning.